CFNetwork crash on iOS 16 devices

Hello, I am getting crashes on iOS 16 devices only regarding CFNetwork. Below is the full crash report. I am not able to reproduce it on my end. I've attached the .txt crash log below and also posted it below.

Any help is appreciated, thank you!

Crashed: com.apple.NSXPCConnection.m-user.com.apple.nsurlsessiond
EXC_BREAKPOINT 0x00000001cfbbecec

7  Foundation                     0x42054 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212
8  Foundation                     0x41f3c -[NSRunLoop(NSRunLoop) runUntilDate:] + 64
9  UIKitCore                      0x4d66a4 -[UIEventFetcher threadMain] + 436
10 Foundation                     0x5b518 __NSThread__start__ + 716
11 libsystem_pthread.dylib        0x16cc _pthread_start + 148
12 libsystem_pthread.dylib        0xba4 thread_start + 8

com.google.firebase.crashlytics.MachExceptionServer
0  App                    	  0x387cfc FIRCLSProcessRecordAllThreads + 393 (FIRCLSProcess.c:393)
1  App                   	  0x3880dc FIRCLSProcessRecordAllThreads + 424 (FIRCLSProcess.c:424)
2  App                    	  0x395be0 FIRCLSHandler + 34 (FIRCLSHandler.m:34)
3  App                     	  0x396400 FIRCLSMachExceptionServer + 521 (FIRCLSMachException.c:521)
4  libsystem_pthread.dylib        0x16cc _pthread_start + 148
5  libsystem_pthread.dylib        0xba4 thread_start + 8

GAIThread
0  libsystem_kernel.dylib         0xda8 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x13a1c mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x13c5c mach_msg_overwrite + 388
3  libsystem_kernel.dylib         0x12ec mach_msg + 24
4  CoreFoundation                 0x7aac4 __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x7bd08 __CFRunLoopRun + 1232
6  CoreFoundation                 0x80eb0 CFRunLoopRunSpecific + 612
7  Foundation                     0x42054 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212
8  Foundation                     0x41ee8 -[NSRunLoop(NSRunLoop) run] + 64
9  App                   	  0x563d00 +[GAI threadMain:] + 64
10 Foundation                     0x5b518 __NSThread__start__ + 716
11 libsystem_pthread.dylib        0x16cc _pthread_start + 148
12 libsystem_pthread.dylib        0xba4 thread_start + 8

com.apple.NSURLConnectionLoader
0  libsystem_kernel.dylib         0xda8 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x13a1c mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x13c5c mach_msg_overwrite + 388
3  libsystem_kernel.dylib         0x12ec mach_msg + 24
4  CoreFoundation                 0x7aac4 __CFRunLoopServiceMachPort + 160
5  CoreFoundation                 0x7bd08 __CFRunLoopRun + 1232
6  CoreFoundation                 0x80eb0 CFRunLoopRunSpecific + 612
7  CFNetwork                      0x257ff0 _CFURLStorageSessionDisableCache + 61088
8  Foundation                     0x5b518 __NSThread__start__ + 716
9  libsystem_pthread.dylib        0x16cc _pthread_start + 148
10 libsystem_pthread.dylib        0xba4 thread_start + 8

CommandHandler
0  libsystem_kernel.dylib         0xda8 mach_msg2_trap + 8
1  libsystem_kernel.dylib         0x13a1c mach_msg2_internal + 80
2  libsystem_kernel.dylib         0x13c5c mach_msg_overwrite + 388
3  libsystem_kernel.dylib         0x12ec mach_msg + 24
4  CaptiveNetwork                 0x9d78 ConnectionGetCommandInfo + 160
5  CaptiveNetwork                 0x7c54 __add_signal_port_source_block_invoke_2 + 244
6  libdispatch.dylib              0x3f88 _dispatch_client_callout + 20
7  libdispatch.dylib              0x7418 _dispatch_continuation_pop + 504
8  libdispatch.dylib              0x1aa58 _dispatch_source_invoke + 1588
9  libdispatch.dylib              0xb518 _dispatch_lane_serial_drain + 376
10 libdispatch.dylib              0xc18c _dispatch_lane_invoke + 384
11 libdispatch.dylib              0x16e10 _dispatch_workloop_worker_thread + 652
12 libsystem_pthread.dylib        0xdf8 _pthread_wqthread + 288
13 libsystem_pthread.dylib        0xb98 start_wqthread + 8

Thread
0  libsystem_kernel.dylib         0x12b0 __workq_kernreturn + 8
1  libsystem_pthread.dylib        0xe44 _pthread_wqthread + 364
2  libsystem_pthread.dylib        0xb98 start_wqthread + 8

Thread
0  libsystem_pthread.dylib        0xb90 start_wqthread + 254

Crashed: com.apple.NSXPCConnection.m-user.com.apple.nsurlsessiond
0  libobjc.A.dylib                0x6cec objc_opt_respondsToSelector + 48
1  libsystem_trace.dylib          0x6480 _os_log_fmt_flatten_object + 248
2  libsystem_trace.dylib          0x41a0 _os_log_impl_flatten_and_send + 1864
3  libsystem_trace.dylib          0x21bc _os_log + 152
4  libsystem_trace.dylib          0x7840 _os_log_impl + 24
5  CFNetwork                      0x10dc08 _CFURLConnectionCopyTimingData + 34880
6  Foundation                     0x64b620 message_handler_error + 360
7  libxpc.dylib                   0x1179c _xpc_connection_call_event_handler + 152
8  libxpc.dylib                   0x11be8 _xpc_connection_mach_event + 1020
9  libdispatch.dylib              0x4048 _dispatch_client_callout4 + 20
10 libdispatch.dylib              0x24104 _dispatch_mach_cancel_invoke + 128
11 libdispatch.dylib              0x21720 _dispatch_mach_invoke + 916
12 libdispatch.dylib              0xb518 _dispatch_lane_serial_drain + 376
13 libdispatch.dylib              0xc1c0 _dispatch_lane_invoke + 436
14 libdispatch.dylib              0x16e10 _dispatch_workloop_worker_thread + 652
15 libsystem_pthread.dylib        0xdf8 _pthread_wqthread + 288
16 libsystem_pthread.dylib        0xb98 start_wqthread + 8

Do you have an Apple crash report for this? If so, please post it here. See Posting a Crash Report for advice on how to do that.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi @eskimo, I was able to find an Apple crash report similar to the crashed thread above. I've attached it below, thanks!

I'm just checking to see if there was ever a resolution to this? I'm seeing this in the crashes too, up to the latest iOS version. It's a very small percentage of users that are seeing this but it's enough (e.g. > 0 😊) to want to fix it.

From the crash reports this is only happening when the app is in the foreground. The app does use a background URLSession but the requests are only made when the app is active. From the state of the other threads this looks to happen just after the app is brought to the foreground.

This isn't something we've been able to recreate while debugging or without a debugger (in case we're hitting some restriction that is disabled when debugging). It's rare enough that there's no clear cause.

We see a few different exceptions causing this, e.g. EXC_BAD_ACCESS (SIGSEGV); KERN_INVALID_ADDRESS at 0x00004a54c4966330 -> 0x00000054c4966330 (possible pointer authentication failure), EXC_BAD_ACCESS (SIGSEGV);KERN_INVALID_ADDRESS at 0x000000006874616e, and EXC_BREAKPOINT (SIGTRAP). I can redact and provide some Apple crash reports if it would be of some use, but they are effectively the same as the one above.

up to the latest iOS version.

Please post a crash report that shows this happening on iOS 17.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Sure thing, I've attached 2.

When locally symbolicated I can see that thread 12 is code that we run in response to UIApplication.willEnterForegroundNotification (after a DispatchQueue.async call). This is what makes me think this is happening not long after the app enters the foreground.

The next crash has a different exception type. Thread 21 is the code run in response to UIApplication.willEnterForegroundNotification.

Thanks Quinn!

Thanks for those up-to-date crash reports.

There are two posters on this thread, JosephDuffy and joseph0427. Earlier you wrote:

I'm seeing this in the crashes too

which suggests you’re not the same person. Which is quite a coincidence. I’m starting to think that CFNetwork might have code like this:

if user.firstName == "Joseph" {
    … corrupt our state …
}

(-:

That’s because AFAICT you folks are the only ones having this problem. Normally when I investigate weird CFNetwork crashes there are a bunch of folks in the same boat, including iOS system components. That’s not the case here, as least AFAICT.

This is what makes me think this is happening not long after the app enters the foreground.

Right. And that gels with the crashing thread’s backtrace. Here’s the backtrace from your first crash report:

Thread 14 name:
Thread 14 Crashed:
0   libobjc.A.dylib         … object_isClass + 16 (objc-class.mm:218)
1   libsystem_trace.dylib   … _os_log_fmt_flatten_object + 424 (format.m:314)
2   libsystem_trace.dylib   … _os_log_impl_flatten_and_send + 1864 (log.c:2672)
3   libsystem_trace.dylib   … _os_log + 168 (log.c:2783)
4   libsystem_trace.dylib   … _os_log_impl + 28 (log.c:2801)
5   CFNetwork               … __46-[__NSURLBackgroundSession 
				setupXPCConnection]_block_invoke_2.23 + 212 (BackgroundSession.mm:0)
6   Foundation              … message_handler_error + 360 (NSXPCConnection.m:855)
7   libxpc.dylib            … _xpc_connection_call_event_handler + 144 (connection.c:830)
8   libxpc.dylib            … _xpc_connection_mach_event + 1156 (connection.c:2453)
9   libdispatch.dylib       … _dispatch_client_callout4 + 20 (object.m:616)
10  libdispatch.dylib       … _dispatch_mach_cancel_invoke + 128 (mach.c:2628)
11  libdispatch.dylib       … _dispatch_mach_invoke + 908 (mach.c:2861)
12  libdispatch.dylib       … _dispatch_lane_serial_drain + 368 (queue.c:3900)
13  libdispatch.dylib       … _dispatch_lane_invoke + 432 (queue.c:3991)
14  libdispatch.dylib       … _dispatch_root_queue_drain_deferred_wlh + 288 (queue.c:6998)
15  libdispatch.dylib       … _dispatch_workloop_worker_thread + 404 (queue.c:6592)
16  libsystem_pthread.dylib … _pthread_wqthread + 288 (pthread.c:2665)
17  libsystem_pthread.dylib … start_wqthread + 8 (:-1)

Frames 0 and 1 suggest that the app has crashed trying to log an NSObject, using the %@ syntax. Frames 2 through 4 are all system log infrastructure. Frame 5 is CFNetwork doing something related to NSURLSession background sessions. And that leads to frame 6, which contains the source line NSXPCConnection.m:855.

I looked up this code and it’s NSXPCConnection calling the connection’s invalidation handler. So, it seems that frame 5 is the invalidation handler for the connection that the NSURLSession uses to talk to its backing daemon (nsurlsessiond [1]). That handler doesn’t actually do anything useful (more on that below) but it does log the message BackgroundSession <UUID> connection to background transfer daemon invalidated, where UUID is a UUID used to keep track of background sessions. It seems that there’s something borked with this UUID, which is the immediate cause of this crash.

However, this raises an interesting question: Why is this invalidation handler being called at all? In XPC parlance:

  • An interruption occurs when the connection fails but it’s worth retrying, for example, when the remote process crashed.

  • An invalidation occurs when it fails and it’s not worth retrying, for example, when the remote peer’s service has been unloaded.

It’s not clear how you’d get an invalidation for a system service, like the one provided by nsurlsessiond. My best guess is that something extra-ordinary is going on, for example, your app is brought to the foreground just as the system is shutting down and hence these services are going away. However, that’s wild speculation on my part.

I don’t think you’re going to be able to make further progress on this without a sysdiagnose log taken shortly after it occurs.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] If you’re curious, you can see how this works on macOS:

% plutil -p /System/Library/LaunchAgents/com.apple.nsurlsessiond.plist
{
  "EnablePressuredExit" => 1
  "EnableTransactions" => 1
  "Label" => "com.apple.nsurlsessiond"
  "MachServices" => {
    "com.apple.nsurlsessiond" => 1
    "com.apple.nsurlsessiond-cachedelete" => 1
    "com.apple.nsurlsessiond-launchondemand" => 1
  }
  "POSIXSpawnType" => "Adaptive"
  "Program" => "/usr/libexec/nsurlsessiond"
  "QueueDirectories" => [
    0 => "~/Library/com.apple.nsurlsessiond"
  ]
}

I didn't even notice we're both called Joseph -- we are 2 different people so I can see that could be confusing 😄

The only other interesting thing I've noticed is that we sometimes get a call to URLSessionTaskDelegate.urlSession(_:task:didCompleteWithError:) with a task that has a description we're not expecting. We set the taskDescription to a file URL's absoluteString and then try to create the URL again with URL.init(string:), but that will fail. Unfortunately I don't have an example of what we actually get (e.g. taskDescription could have been corrupted in some way) but it might be a clue to the underlying problem.

We've never seen this during debugging, which is our primary use of the app. If we ever find a way to recreate this I will get a sysdiagnose.

We have been thinking about opening a DTS for this. It sounds like there's not much more that could be looked at so this might not be worth it?

but it might be a clue to the underlying problem.

Yeah, that does sound suspicious. It’d be interesting to see what the description was in that case, so if you can add a log point that includes it that’d be grand.

We have been thinking about opening a DTS for this. It sounds like there's not much more that could be looked at so this might not be worth it?

Agreed.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you for your support on this Quinn. I just so happened to run in to the scenario where the taskDescription is nil this morning. Just before it I see the following logged (this looks to be by iOS):

Task <39EBDA50-1653-4B16-B5FC-FBC1CDE9B130>.<462> finished with error [-1000] Error Domain=NSURLErrorDomain Code=-1000 "bad URL" UserInfo={_kCFStreamErrorCodeKey=22, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundUploadTask <39EBDA50-1653-4B16-B5FC-FBC1CDE9B130>.<462>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "BackgroundUploadTask <39EBDA50-1653-4B16-B5FC-FBC1CDE9B130>.<462>",
    "LocalUploadTask <39EBDA50-1653-4B16-B5FC-FBC1CDE9B130>.<462>"
), NSLocalizedDescription=bad URL, _kCFStreamErrorDomainKey=1, NSErrorFailingURLStringKey=https://my-subdomain.on.company.com/event/pack, NSErrorFailingURLKey=https://my-subdomain.on.company.com/event/pack}

This is the same error passed in to URLSessionTaskDelegate.urlSession(_:task:didCompleteWithError:).

I can't see anything wrong with this URL though. The value of task.currentRequest?.url?.absoluteString is https://my-subdomain.on.company.com/event/pack so URL at least thinks it's valid.

Until I can recreate the actual crash I don't know if this scenario is a red herring or not.

In an attempt to provide as much as I can here I've already attached the output of p task.

I can't see anything wrong with this URL though.

Right. It sounds like a CFNetwork has got itself tied up in knots there.

If you’re going to continue to work on reproducing this, I recommend that you install the CFNetwork and Network Diagnostics debug profiles on your test devices. That’ll make sure your sysdiagnose log has the most info available for analysis.

Get those profiles from our Bug Reporting > Profiles and Logs.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

CFNetwork crash on iOS 16 devices
 
 
Q